Using transfer learning for classification of gait pathologies

Different diseases can affect an individual’s gait in different ways and, therefore, gait analysis can provide important insights into an individual’s health and well-being. Currently, most systems that perform gait analysis using 2D video are limited to simple binary classification of gait as being either normal or impaired. While some systems do perform gait classification across different pathologies, the reported results still have a considerable margin for improvement. This paper presents a novel system that performs classification of gait across different pathologies, with considerably improved results. The system computes the walking individual’s silhouettes, which are computed from a 2D video sequence, and combines them into a representation known as the gait energy image (GEI), which provides robustness against silhouette segmentation errors. In this work, instead of using a set of handcrafted gait features, feature extraction is done using the VGG-19 convolutional neural network. The network is fine-tuned to automatically extract the features that best represent gait pathologies, using transfer learning. The use of transfer learning improves the classification accuracy while avoiding the need of a very large training set, as the network is pre-trained for generic image description, which also contributes to a better generalization when tested across different datasets. The proposed system performs the final classification using linear discriminant analysis (LDA). Obtained results show that the proposed system outperforms the state-of-the-art, achieving a classification accuracy of 95% on a dataset containing gait sequences affected by diplegia, hemiplegia, neuropathy and Parkinson’s disease, along with normal gait sequences.info:eu-repo/semantics/acceptedVersio

View-invariant gait recognition exploiting spatio-temporal information and a dissimilarity metric

In gait recognition, when subjects do not follow a known walking trajectory, the comparison against a database may be rendered impossible. Some proposed solutions rely on learning and mapping the appearance of silhouettes along various views, with some limitations caused for instance by appearance changes (e.g. coats or bags). The present paper discusses this problem and proposes a novel solution for automatic viewing angle identification, using minimal information computed from the walking person silhouettes, while being robust against appearance changes. The proposed method is more efficient and provides improved results when compared to the available alternatives. Moreover, unlike most state-of-the- art methods, it does not require a training stage. The paper also discusses the use of a dissimilarity metric for the recognition stage. Dissimilarity metrics have shown interesting results in several recognition systems. This paper also attests the strength of a dissimilarity-based approach for gait recognition.info:eu-repo/semantics/acceptedVersio

View-invariant gait recognition system using a gait energy image decomposition method

Gait recognition systems can capture biometrical information from a distance and without the user's active cooperation, making them suitable for surveillance environments. However, there are two challenges for gait recognition that need to be solved, namely when: (i) the walking direction is unknown and/or (ii) the subject's appearance changes significantly due to different clothes being worn or items being carried. This study discusses the problem of gait recognition in unconstrained environments and proposes a new system to tackle recognition when facing the two listed challenges. The system automatically identifies the walking direction using a perceptual hash (PHash) computed over the leg region of the gait energy image (GEI) and then compares it against the PHash values of different walking directions stored in the database. Robustness against appearance changes are obtained by decomposing the GEI into sections and selecting those sections unaltered by appearance changes for comparison against a database containing GEI sections for the identified walking direction. The proposed recognition method then recognises the user using a majority decision voting. The proposed view-invariant gait recognition system is computationally inexpensive and outperforms the state-of-the-art in terms of recognition performance.info:eu-repo/semantics/acceptedVersio

Gait recognition in the wild using shadow silhouettes

Gait recognition systems allow identification of users relying on features acquired from their body movement while walking. This paper discusses the main factors affecting the gait features that can be acquired from a 2D video sequence, proposing a taxonomy to classify them across four dimensions. It also explores the possibility of obtaining users’ gait features from the shadow silhouettes by proposing a novel gait recognition system. The system includes novel methods for: (i) shadow segmentation, (ii) walking direction identification, and (iii) shadow silhouette rectification. The shadow segmentation is performed by fitting a line through the feet positions of the user obtained from the gait texture image (GTI). The direction of the fitted line is then used to identify the walking direction of the user. Finally, the shadow silhouettes thus obtained are rectified to compensate for the distortions and deformations resulting from the acquisition setup, using the proposed four-point correspondence method. The paper additionally presents a new database, consisting of 21 users moving along two walking directions, to test the proposed gait recognition system. Results show that the performance of the proposed system is equivalent to that of the state-of-the-art in a constrained setting, but performing equivalently well in the wild, where most state-of-the-art methods fail. The results also highlight the advantages of using rectified shadow silhouettes over body silhouettes under certain conditions.info:eu-repo/semantics/acceptedVersio

Sparse error gait image: a new representation for gait recognition

The performance of a gait recognition system is very much related to the usage of efficient feature representation and recognition modules. The first extracts features from an input image sequence to represent a user's distinctive gait pattern. The recognition module then compares the features of a probe user with those registered in the gallery database. This paper presents a novel gait feature representation, called Sparse Error Gait Image (SEGI), derived from the application of Robust Principal Component Analysis (RPCA) to Gait Energy Images (GEI). GEIs obtained from the same user at different instants always present some differences. Applying RPCA results in low-rank and sparse error components, the former capturing the commonalities and encompassing the small differences between input GEIs, while the larger differences are captured by the sparse error component. The proposed SEGI representation exploits the latter for recognition purposes. This paper also proposes two simple approaches for the recognition module, to exploit the SEGI, based on the computation of a Euclidean norm or the Euclidean distance. Using these simple recognition methods and the proposed SEGI representation gait recognition, results equivalent to the state-of-the-art are obtained

A spatiotemporal deep learning approach for automatic pathological Gait classification

Human motion analysis provides useful information for the diagnosis and recovery assessment of people suffering from pathologies, such as those affecting the way of walking, i.e., gait. With recent developments in deep learning, state-of-the-art performance can now be achieved using a single 2D-RGB-camera-based gait analysis system, offering an objective assessment of gait-related pathologies. Such systems provide a valuable complement/alternative to the current standard practice of subjective assessment. Most 2D-RGB-camera-based gait analysis approaches rely on compact gait representations, such as the gait energy image, which summarize the characteristics of a walking sequence into one single image. However, such compact representations do not fully capture the temporal information and dependencies between successive gait movements. This limitation is addressed by proposing a spatiotemporal deep learning approach that uses a selection of key frames to represent a gait cycle. Convolutional and recurrent deep neural networks were combined, processing each gait cycle as a collection of silhouette key frames, allowing the system to learn temporal patterns among the spatial features extracted at individual time instants. Trained with gait sequences from the GAIT-IT dataset, the proposed system is able to improve gait pathology classification accuracy, outperforming state-of-the-art solutions and achieving improved generalization on cross-dataset tests.info:eu-repo/semantics/publishedVersio

Remote Gait type classification system using markerless 2D video

Several pathologies can alter the way people walk, i.e., their gait. Gait analysis can be used to detect such alterations and, therefore, help diagnose certain pathologies or assess people’s health and recovery. Simple vision-based systems have a considerable potential in this area, as they allow the capture of gait in unconstrained environments, such as at home or in a clinic, while the required computations can be done remotely. State-of-the-art vision-based systems for gait analysis use deep learning strategies, thus requiring a large amount of data for training. However, to the best of our knowledge, the largest publicly available pathological gait dataset contains only 10 subjects, simulating 5 types of gait. This paper presents a new dataset, GAIT-IT, captured from 21 subjects simulating 5 types of gait, at 2 severity levels. The dataset is recorded in a professional studio, making the sequences free of background camouflage, variations in illumination and other visual artifacts. The dataset is used to train a novel automatic gait analysis system. Compared to the state-of-the-art, the proposed system achieves a drastic reduction in the number of trainable parameters, memory requirements and execution times, while the classification accuracy is on par with the state-of-the-art. Recognizing the importance of remote healthcare, the proposed automatic gait analysis system is integrated with a prototype web application. This prototype is presently hosted in a private network, and after further tests and development it will allow people to upload a video of them walking and execute a web service that classifies their gait. The web application has a user-friendly interface usable by healthcare professionals or by laypersons. The application also makes an association between the identified type of gait and potential gait pathologies that exhibit the identified characteristics.info:eu-repo/semantics/publishedVersio

Estimation and validation of temporal gait features using a markerless 2D video system

Background and Objective: Estimation of temporal gait features, such as stance time, swing time and gait cycle time, can be used for clinical evaluations of various patient groups having gait pathologies, such as Parkinson’s diseases, neuropathy, hemiplegia and diplegia. Most clinical laboratories employ an optoelectronic motion capture system to acquire such features. However, the operation of these systems requires specially trained operators, a controlled environment and attaching reflective markers to the patient’s body. To allow the estimation of the same features in a daily life setting, this paper presents a novel vision based system whose operation does not require the presence of skilled technicians or markers and uses a single 2D camera. Method: The proposed system takes as input a 2D video, computes the silhouettes of the walking person, and then estimates key biomedical gait indicators, such as the initial foot contact with the ground and the toe off instants, from which several other temporal gait features can be derived. Results: The proposed system is tested on two datasets: (i) a public gait dataset made available by CASIA, which contains 20 users, with 4 sequences per user; and (ii) a dataset acquired simultaneously by a marker-based optoelectronic motion capture system and a simple 2D video camera, containing 10 users, with 5 sequences per user. For the CASIA gait dataset A the relevant temporal biomedical gait indicators were manually annotated, and the proposed automated video analysis system achieved an accuracy of 99% on their identification. It was able to obtain accurate estimations even on segmented silhouettes where, the state-of-the-art markerless 2D video based systems fail. For the second database, the temporal features obtained by the proposed system achieved an average intra-class correlation coefficient of 0.86, when compared to the "gold standard" optoelectronic motion capture system. Conclusions: The proposed markerless 2D video based system can be used to evaluate patients’ gait without requiring the usage of complex laboratory settings and without the need for physical attachment of sensors/markers to the patients. The good accuracy of the results obtained suggests that the proposed system can be used as an alternative to the optoelectronic motion capture system in non-laboratory environments, which can be enable more regular clinical evaluations.info:eu-repo/semantics/acceptedVersio

Shadow type identification for gait recognition using shadows

\Using features acquired from the shadow cast by a walking person can be an alternative for gait recognition whenever the person’s body is occluded, such as when capturing images from an overhead position. However, the shadow, depending on the light source characteristics, can be cast as a blob with no distinguishing characteristics. Most state-of-the-art methods fail in the presence of such “diffused” shadows. Thus, this paper presents a novel method to identify the type of shadow cast by a person. The proposed method generates a histogram of the intensity ratio between foreground and background areas, whose analysis allows identifying the type of shadow cast by the person. The proposed method is very promising, achieving a 90% correct shadow type identification with the dataset tested.info:eu-repo/semantics/publishedVersio

Automatic Classification of Gait Impairments Using a Markerless 2D Video-Based System

Systemic disorders affecting an individual can cause gait impairments. Successful acquisition and evaluation of features representing such impairments make it possible to estimate the severity of those disorders, which is important information for monitoring patients’ health evolution. However, current state-of-the-art systems perform the acquisition and evaluation of these features in specially equipped laboratories, typically limiting the periodicity of evaluations. With the objective of making health monitoring easier and more accessible, this paper presents a system that performs automatic detection and classification of gait impairments, based on the acquisition and evaluation of biomechanical gait features using a single 2D video camera. The system relies on two different types of features to perform classification: (i) feet-related features, such as step length, step length symmetry, fraction of foot flat during stance phase, normalized step count, speed; and (ii) body-related features, such as the amount of movement while walking, center of gravity shifts and torso orientation. The proposed system uses a support vector machine to decide whether the observed gait is normal or if it belongs to one of three different impaired gait groups. Results show that the proposed system outperforms existing markerless 2D video-based systems, with a classification accuracy of 98.8%